feat(vllm-tensorizer): Bump vLLM to v0.20.2 on CUDA 13.2 / Ubuntu 24.04#160
Conversation
…-containers into jperlman/vllm0.20.2
|
@JustinPerlman Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/25751418629 |
Fair enough lol |
|
Pure 13.2, no matrix with 12.9? 🫣I would really like having both options…if it’s a giant pain on vllm side it’s fine, but I think you then need to validate this actually works on b40/rtxp6000 with latest supported/installed drivers cw ships |
|
I am still not aware of a cuda + driver combo that has decent support and works as expected, but haven’t followed too closely lately |
|
@JustinPerlman Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/25919982852 |
|
@JustinPerlman Build complete, success: https://github.com/coreweave/ml-containers/actions/runs/25919982852 |
There was a problem hiding this comment.
I'd personally suggest to not repeat yourself as much in this config file and to construct more parts of this dynamically, like the tag suffix, but not a hard requirement.
| rm -f /usr/lib/python3.*/EXTERNALLY-MANAGED && \ | ||
| python3 -m pip install -U --no-cache-dir pip packaging 'setuptools>=77.0.3,<81.0.0' wheel setuptools_scm regex build |
There was a problem hiding this comment.
rm -f /usr/lib/python3.*/EXTERNALLY-MANAGED and pip/setuptools installation and upgrading is already handled by the torch image, so you don't need to repeat those bits here.
| apt-get install -y --no-install-recommends curl libsodium23 libnuma-dev && \ | ||
| apt-get purge -y python3-jwt && \ | ||
| apt-get clean && \ | ||
| rm -f /usr/lib/python3.*/EXTERNALLY-MANAGED |
There was a problem hiding this comment.
Same comment as before: this rm is already handled by the base image.
| RUN apt-get -qq update && apt-get install -y --no-install-recommends curl libsodium23 libnuma-dev && apt-get clean | ||
| RUN apt-get -qq update && \ | ||
| apt-get install -y --no-install-recommends curl libsodium23 libnuma-dev && \ | ||
| apt-get purge -y python3-jwt && \ |
There was a problem hiding this comment.
What's that apt-get purge -y python3-jwt for? 👀
4 participants
Summary
v0.20.2-cuda13.2.1-ubuntu24.04v0.20.2-cuda12.9.1-ubuntu24.04Ubuntu 24.04 compatibility fixes
python3-pipfrom apt inbuilder-baseand addrm -f /usr/lib/python3.*/EXTERNALLY-MANAGEDbefore pip bootstrap — on Ubuntu 24.04, apt-installed pip has no RECORD file and blocks pip self-upgradepython3-jwtin the finalbasestage before pip installs — same root cause: Debian-managed PyJWT has no RECORD file and blocks vLLM's dependency resolutioncuda-pythonversion spec from~=${CUDA_VERSION}to~=${CUDA_VERSION%.*}— patch-level CUDA versions (e.g.13.2.1) don't match available cuda-python releases; strip to major.minorwheelpackage inlmcache-builderand restore it tobuilder-basepip installRelevant information: vllm-project/vllm@6c964bd